take2 on #11536 #11350

Partial unfold means that we want to skip unfolding of nodes when it's possible to do so.

Node state for account represented by fields [balance/nonce/codeHash] and for storage by it's value.
By default node is not loaded with state. During folding, if needed, state parts are loaded.

Skip is possible in following cases (having memoised state hash already exists):

• no changes to node state (balance/nonce/code/storage was not changed)
• node preserves it's position in trie: keeps depth, keeps nibble
• no subtree updates (eg. no storage updates for account or no extension split due to insertion)

There is still a Q which state hashes are worth keeping and which are not. In bor experiments we've seen that top-level hashes of storages are changing quite frequently, as well as hashes of accounts with lot's of storage keys in it.

This PR totally reworks mechanism of rebuilding commitment files when all other domains exists and have same ranges.

• Processes big shards with smaller parts (eg 0-1024.kv is 8 shards by 128 steps)
• shards become merged when range is finished (from example above, steps will be merged into 0-1024, when all 8 shards are ready (madv friendly way))
• does not write commitment into db, dumps file directly on disk

If cancelled during rebuilt, have to remove all shards which doesn't have matching range to other domains.
In example, accounts are [0-1024, 1024-1536, ..] and commitment are [0-1024, 1024-1152..] you have to do remove commitment files of steps > 1024 (their range does not match to account 1024-1536, rebuilt will be incorrect and there is no protection yet for that case.

To rebuild commitment:

$ make integration && ./build/bin/integration commitment_rebuild --datadir <> --chain <>

To be done:

• squeezing commitment
• move this subfunction into integration commitment along with analysis tool for commitment files.